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Abstract — The cognitive interference channel with confidential 
messages (CICC) proposed by Liang et. al. is investigated. When 
the security is considered in coding systems, it is well known 
that the sender needs to use a stochastic encoding to avoid the 
information about the transmitted confidential message to be 
leaked to an eavesdropper. For the CICC, the trade-off between 
the rate of the random number to realize the stochastic encoding 
and the communication rates is investigated, and the optimal 
trade-off is completely characterized. 

Index Terms — Cognitive Interference Channel, Confidential 
Messages, Randomness Constraint, Stochastic Encoder, Super- 
position Coding 



I. Introduction 

Cognitive radio has attracted considerable attention recently, 
for it can improve the spectrum efficiency of wireless networks 
IT]. In information theoretical study of the cognitive radio, it 
is usually modeled by a interference channel called cognitive 
interference channel (CIC), in which the cognitive transmitter 
can non-causally know the other transmitter's message ||2], 
lEl, H, Q. We consider the (CIC) model investigated by 
Jiang et. al. ||6|, Zhong et. al. |[7), and Liang et. al. fS), in 
which one receiver needs to decode both messages. Especially 
as in 18], we also consider the security, i.e., the message 
sent by the cognitive transmitter must be kept secret from 
one of the receivers. We call this problem the cognitive 
interference channel with confidential messages (CICC). The 
coding system investigated in this paper is described in Fig.[T] 

When the security is considered, it is well known that 
the sender needs to use a stochastic encoder to avoid the 
information about the transmitted confidential message to be 
leaked to the eavesdropper Eve. The stochastic encoder is 
usually realized by preparing a dummy random number in 
addition to the intended messages and by encoding them to 
a transmitted signal by a deterministic encoder Furthermore, 
random numbers are also needed to realize the coding tech- 
nique called channel prefixing. 

In literatures of information theoretic security (eg. (0, ifTol . 
ifTTl ). the random number has been regarded as free resource, 
and the amount of the random number used in the stochastic 
encoding has been paid no attention. However in practice, 
the random number is quite precious resource. For example, 
generation rates of any existing true random number generators 
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are not as fast as communication rates of wireless networks 
||T2J . Although the random number generator equipped in the 
forthcoming Intel's CPU can generate the random number as 
fast as 3Gbps llT3l . the communication rate of the new IEEE 
wireless communication standard is said to be over Gbps jT?] . 
Thus, the random number should be regarded as at least as 
precious as communication resources. For this purpose, we 
formulate the problem of the CICC by randomness constrained 
stochastic encoder, and completely characterize the capacity 
region of this new problem. We assume that the non-cognitive 
transmitter, Charlie, only uses a deterministic encoding. This 
assumption seems natural because Charlie only observes the 
common message, and the common message need not to be 
kept secret. 

The present problem to consider the CICC by the ran- 
domness constrained stochastic encoder is an extension of 
the authors' series of works. In iflSl . the authors investigated 
the capacity region of the relay channel with confidential 
messages for the completely deterministic encoder, and the 
capacity region of the broadcast channel with confidential 
messages (BCC) for the completely deterministic encoder was 
characterized as a corollary. In llT6l . the authors completely 
characterized the capacity region of the BCC by the random- 
ness constrained stochastic encoder The problem formulation 
in this paper is the extension of that in llT6l to the CIC, and 
more involved coding techniques are needed. 

Since the security criterion employed in this paper is slightly 
different from that in ||8], it should be remarked. In (|8], the 
cognitive transmitter, Alice, sends two kinds of messages, the 
common message and the confidential message, and the level 
of secrecy of the confidential message was evaluated by the 
equivocation rate. In this paper, Alice sends three kinds of 
messages, the common message, the private message, and the 
confidential message. The role of the common message is the 
same as that in IH. The private message is supposed to be 
decoded by one of the receiver. Bob, and we do not care 
whether Eve can decode the private message or not. On the 
other hand, the confidential message is supposed to be decoded 
by Bob, and it must be kept completely secret from Eve. The 
secrecy of the confidential message is evaluated by the so- 
called strong security criterion ifTTI . llTSl . As a byproduct, our 
direct coding theorem is stronger than that in ||8], i.e., our 
theorem states the strong secrecy. 

The reason we do not use the equivocation rate formulation 
is as follows. In the conventional equivocation rate formula- 
tion, if the rate of dummy randomness is not sufficient, a part 
of the confidential message is sacrificed to make the other part 
completely secret and the rate of the completely secret part 
corresponds to the equivocation rate. We think that the rates 
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Fig. L The coding system investigated in this paper. Alice sends common 
message Kn, private message L„, and confidential message Sn by using 
a deterministic function /„ and a limited amount of dummy randomness 
An- Charlie also sends a signal J which is a deterministic function of the 
common message Kn- The common message is supposed to be decoded by 
both Bob and Eve. The private message is supposed to be decoded by Bob, 
and we do not care whether Eve can decode the private message or not. The 
confidential message is supposed to be decoded by Bob, and it must be kept 
completely secret from Eve. 



of sacrificed part and completely secret part become clearer 
by employing our formulation. 

The rest of this paper is organized as follows. In Section 
Hn the problem formulation is explained and main results are 
presented. In Section |III1 the proof of the main theorem is 
presented. Some technical arguments are presented in Appen- 
dices. 

II. Problem Formulation and Main Results 

Let Py \X1X2 ^nd Pz\XiX2 be two channels with common 
input alphabets Xi x X2 and output alphabets y and Z 
respectively. Throughout the paper, the alphabets are assumed 
to be finite though we do not use finiteness of the alphabet 
except cardinality bonds on auxiliary random variables. 

Let /C„ be the set of the common message, be the set 
of the private message, and Sn be the set of the confidential 
message. The common message is supposed to be decoded 
by both Bob and Eve. The private message is supposed to 
be decoded by Bob, and we do not care whether Eve can 
decode the private message or not. The confidential message 
is supposed to be decoded by Bob, and it must be kept 
completely secret from Eve. 

Typically, Alice use a stochastic encoder to make the confi- 
dential message secret from Eve, and it is practically realized 
by using a uniform dummy randomness on the alphabet An- 
When the size \An\ of dummy randomness is infinite, any 
stochastic encoder from /C„ x £„ x <S„ to A"" can be simulated 
by a deterministic encoder /„ : /C„ x Cn x 
But we are interested in the case with bounded size \An\ in 
this paper. In this paper, we assume that Charlie only use a 



deterministic encoder : A^„ X2- 

Bob's decoder is defined by function gn ■ 3^" 
Sn and the error probability is defined as 

Perr ifn i fn i 9n ) 

= E E E E 



ICn X £,-,, 



1 



YlXiX 



,{y \In{kn, (-m Sn-i (In)-: fniJ^n)) 



i[5«(y") ^ (fcn,Cs«)] 



(1) 



where 1[ ] is the indicator function. Eve's decoder is defined 
by function : Z" — J> JCn and the error probability 
Perrifm 4'n) IS defined in a similar manner as Eq. ([Hi. 
Let 



P. 



E E E 

kr,eK„ e„eC„ a„GA 



1 



l^n I I \ \ 
|/n(^n5 ^127 ^n) ■> fn(.^n)^ 7 



pn 



be the output distributions of the channel Pz\XiX2 
paper, we consider the security criterion given by 



In this 



D{fnJ'n) ■■= D{P,^^4Ps,^ X P^„) 

= liSn;Z"), 

where is the divergence, and /(■;■) is the mutual 

information [19] ■ The coding system investigate in this paper 
is depicted in Fig. [l] 

In this paper, we are interested in the trade-off among the 
rate the dummy randomness, and the rates of the common, 
private, and confidential messages. 

Definition 1: The rate quadruple {Rd, Ro, Ri, Rs) is said to 
be achievable if there exists a sequence of Alice's deterministic 
encoder /„ : /C,i x £„ x iS„ x ^„ — > A"", Charlie's deterministic 
encoder f^-.K-n^ X2, Bob's decoder g„ : 3^" /Cn x £„ x 
Sn, and Eve's decoder 0„ : Z" ICn such that 



lim Perrifn, fn^gn) = 0, 

n— >-oo 

lim Perrifn, fniK) = 0, 
n— >-oo 

lim DifnJ^) = 0, 

n—^QO 

1 



lim sup - log I ^„ I < Rd, 
lim inf — log | /C„ 



lim -log|£„ 

n—>oo Ji 

lim inf — log \Sn 



> Ro, 
~ R\, 

> Rs. 



(2) 
(3) 
(4) 

(5) 

(6) 

(7) 

(8) 



Then the achievable region TZ is defined as the set of all 

achievable rate quadruples. 

The following is our main result in this paper 

Theorem 2: Let TZ* be a closed convex set consisting 

of those quadruples {Rd, Rq, Ri, Rs) for which there exist 

auxiliary random variables {U, V) such that 



{U,V) ^{Xi,X2)^ {Y,Z) 
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and 

Ro < mm[I{U,X2;Y),I{U,Xr,Z)], (9) 

Ri+Rs < I{U,V;Y\X2), (10) 

Ra + Ri+Rs < I{V;Y\U,X2) 

+ mm[IiU, ; r ) , /(£/, X2 ; Z)] ,( 1 1) 

Rs < I{V;Y\U,X2)- I{V;Z\U, X2)il2) 

Ri + Rd > I{Xi;Z\U,X2), (13) 

Rd > I{Xi;Z\U,V,X2). (14) 

Then we have TZ = TZ* . Moreover, it may be assumed that 
the ranges of U and V may be assumed to satisfy 

\U\ < \Xi\\X2\+3, 
\V\ < \X1f\X2f +4\Xl\\X2\+3■ 
Proof: See Section HUl ■ 
Remark 3: As we will find in the achievability proof of the 
main theorem, the private message can be used as dummy 
randomness to protect the confidential message from Eve. 
Thus, if we define the achievability rate region TZ by replacing 
Eq. © with 

lim inf — log |£„ | > i?i , 

region TZ is broader than region TZ. Indeed, 7^ is a closed 
convex set consisting of those quadruple {Rd, Ro, Ri, Rs) for 
which there exist auxiliary random variables {U, V) satisfying 
the same conditions as Theorem |2] except Eq. ( fTsT l. 

Remark 4: Eq. (fT4l i means that here is a certain amount of 
dummy randomness that cannot be substituted by the private 
message. Note that the difference between the private message 
and the dummy randomness is whether Bob needs to decode 
it or not. 

When there is no randomness constraint, region 

T^oD = {(^0, Ri,Rs) ■ ^Rd > s.t. {Rd, Ro, Ri,Rs) G TZ} 

coincide with the result obtained by Liang et. al. H]. 

Corollary 5: (|8i) Region T^oo is a closed convex set con- 
sisting of those triplet {Ro, Ri, Rs) for which there exist 
auxiliary random variables {U, V) such that 

{U,X2) ^ Xi, 

{U,V) O {XuX2) o {Y,Z) 

and 

Ro < Tnin[I{U,X2;Y),I{U,X2;Z)], 
Ri+Rs < I{U,V;Y\X2), 
Ro + Ri+Rs < I{V;Y\U,X2) 

+ mm[I{U,X2;Y),I{U,X2;Z)], 
Rs < I{V;Y\U,X2)-I{V;Z\U,X2). 

III. Proof of Main Results 

A. Proof of Direct Part of Theorem |2] 

The direct part of Theorem |2] follows from the following 
Corollary |7] and Lemma |8] 



We first show the following. 

Lemma 6: Let T?.*^'"^ be a closed convex set consisting of 
those quadruples {Rd, Ro, Ri,Rs) for which there exist ri > 
and auxiliary random variables {U, V) such that 

{U,X2)^V ^ X^, 

{U,V) ^{X,,X2)^ {Y,Z) 

and 



Ro 


+ ri 


< 


I{U,X2;Z), 


Ri-n- 


^Rs 


< 


I{V;Y\U,X2), 


Ri- 


^Rs 


< 


I{U,V;Y\X2), 


Ro + Ri - 


hi?. 


< 


I{U,V,X2;Y), 


Ri 


- ri 


> 


I{V;Z\U,X2), 




Rd 


> 


I{X^;Z\U,V,X2) 



Then we have 7^(™) C TZ. 

Proof: See Section UlLB] ■ 
We note the following observation. From the definition of 
the problem, if 



{Rd ~ Td, Ro, Ri 



rd,Rs 



£ TZ 



for some r^, > 0, then we also have {Rd, Ro,Ri, Rs) G T^- 
Thus, Lemma |6] implies the following corollary. 

Corollary 7: Let 7?.''"' be a closed convex set consisting 
of those quadruples {Rd, Ro, Ri, Rs) for which there exist 
fijrdjTs > and {U, V) such that 

{U,X2)^V ^Xi, 

{U,V) ^{Xi,X2)^ {Y,Z) 

and 





Ro ' 


f n 


< 


I{U,X2;Z), 


Ri-n^ 


-Td^ 


-Rs 


< 


I{V;Y\U,X2), 


Ri 4 


-Td^ 


-Rs 


< 


I{U,V;Y\X2), 


Ro + -Ri -\ 


-Td^ 


-Rs 


< 


I{U,V,X2;Y), 


Ri-n 


- rs - 


f 


> 


I{V-Z\U,X2), 




Rd 


- rd 


> 


I{Xi;Z\U,V,X2) 



Then we have Tt**"' C TZ. 

By using the Fourier-Motzkin elimination, we can also show 
the following. 

Lemma 8: We have 

TZ* C TZ'-'"\ 

Proof: See Appendix ICl ■ 

B. Proof of Lemma \6\ 

For a while, we consider the case with n = 1 and omit the 
superscript and subscript to simplify the notation. We first split 
the private message as £ = I x J7. For each common message 
fc G /C, we randomly generate codeword X2k according to 
distribution Px2- We denote such a code Cq. For each k and 
each i G I, we randomly generate codeword Uki according 
to distribution Pmx2{'\x2k)- We denote such a code Ci. For 
each {k, i) and for each (j, s) G ^7 x 5, we randomly generate 
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codeword Vkijs according to distribution Pv\ux^{'\uki,X2k)- 
We denote such a code C2. For each {k,i,j) and for each 
a ^ A, we randomly generate codeword xikijsa according to 
distribution Px;^\v{^\vkijs)■ We denote such a code C3. 



Let 




r„ - \ 


r 

{u,X2,z) : 


- \ 


{u,v,x2,y) 




{u,v,X2,y) 




{u,v,x2,y) 



Pz\UX2{z\u,X2) 

Pz{z) 

. PY\UVX2iyW,V,X2) 

PY\UX2{y\u,X2) 
. PY\UVX2{y\u,V,X2) 
PY\X2{y\x2) 
PY\UVX2iyW,V,^2) 
PYiv) 



and let T = 7i n 72 n 73. Eve decodes only fc by using the 
indirect decoding proposed in ll20l . Eve's decoding region is 
defined by 

Vk = 

|z : 3i {u^k,X2i,z) e To,yk ^ k yi {u-^~^, x^j^, z) ^ 7B| , 

i.e., = fc if z £ Vk- Bob decodes {k,i,j,s). Bob's 

decoding region is defined by 

'Dkijs = I?/ : {uki,Vkijs,X2k,y) G T, 

V(fc,«, j,s) 7^ {k,i,j,s) {Uf,i,Vf^i-.,x^yy) i r| , 

i-e-, 3(2/) = s) if y e 

Then we have the following. 
Lemma 9: We have 

EC0C1C2C3 [-Perrl/,.?)] 

< Puvx2Y{r^l + Puvx2y{V) + Puvx2Y{Ti) 

+ \J\\S\e-^' + |Z||j7||5|e-"^ + |/C| |Z| | J7| |5|e-"% 

(15) 

EC0C1C2C3 [^'err(/, f/-)] 

< Pc/x.z(ro^) + |/C||I|e-"«, (16) 



and 



< _i_eV'(e|Pz|XiX2,Pxi|V,Pc/VX2) 



I ^ „i'(S'\Pziiivx^ ,Pv\ux2 ^Pux^) ('\n\ 



where 



'>i^iP\Pz\XiX2,Pxi\V,PuVX2) 

= log ^ P(7yx2(w,«,a;2)^ 

^Px^\v{xl\v)Pz\X^X2{zW:X2Y' 



and 

■0 I -Pz I C/ , I C/X2 , ^'(7X2 ) 

= log^P[/X2(u,a;2)^ 

Pv\ux2 {v\u, X2)Pz\uvx2 2:2)^+'' j 

-Pz|c/X2(2^l",a;2)^''. 

Proof: See Appendix iBl ■ 
We apply Lemma |9] for asymptotic case. For 

{Rd,Ro,RiRs) e Te'"') and arbitrary small 5 > 

we set \K.n\ = Le"(-^«-'^')J, |Z„| = [e"('-i-'5)j 

IA.I - Le"(^-^+2^'J, ao = /(C/,X2;Z) - (5 
ai = /(F;r|[/,X2) - (5, a2 = I(U,V;Y\X2) - (5 
as = I{U, V, X2; Y) - S. Then, 

|^„||I„||J„||5„|e-"^" < e-"(^(^^^'^^^^)-^«-^-^-^=+3^), 
|/C„||J„|e-"°" < e-"(^(^'-^^'^'-"«-''i+*) 

converge to asymptotically. Furthermore, by the law of large 
numbers, PSvx2YirCn\ PuVX2YiT2%), PdvX2Y{V,nl 
and Pijx^zO'on) ^1^° converge to asymptotically. 
Since 

^-'(01^^1x1X2,^^X11^, PUVX2) = nx,; Z\U, V, X2) 
there exists Oq > such that 

4'i(^o\Pz\XiX2iPxi\V, PUVX2) 



< I{Xi;Z\U,V,X2) + S <Rd + 5, 



which implies 



Thus, 



log \An I + lJj{Oo\Pz\XiX2 , Pxi\v, PUVX2 ) < 



^mp(9o\Pz\XiX2 ,Pxj^\v,Puvx2) 



exponentially converges to 0. Similarly, since 

^'iO\Pz\uvx2,Pv\ux2,Pux2) ^ I(y;Z\U,X2) 
there exists 0'^^ > such that 

•0(^0 I Pz\UVX2 , Pv\ux2 , Pux2 ) 

< IiV;Z\U,X2) + 5 <Ri-ri+S, 
which implies 

''^l0g\Jn\+^{e'„\PzluVX2,Pv\UX2,PuX2) < S- 



n 
Thus 



1 



7-e 



n4'(e'„\Pz\uvx2^Pvlux2 ^Pux2) 



Z\UVX2 



{z\u,V,X2)~ 



exponentially converges to asymptotically. This completes a 
proof of the lemma. ■ 
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C. Proof of Converse Part of Theorem |2] 

Suppose that Ro,Ri,Rs) £ TZ. Then, for arbitrary 7 > 
0, there exists n such that 





n{Ro 


-7) 


< 


log|/C„|, 


n(i?i 


+ Rs 


-7) 


< 


\0g\Cn\\Sn\, 


n{RQ + Ri 


+ Rs 


-7) 


< 


log /C„ 5„ 




n{Rs 


-7) 


< 


log|<S„|, 


n{Ri 


+ Rd 


+ 7) 


> 


log |/:„||Aj|, 




n{Rd 


-7) 


> 


loglAJ. 



By combining these inequalities with the following Lemma [TOl 
and Lemma[TTl we have the converse part of the theorem. The 
statement about the range size of U and V can be proved in 
the same manner as |8]. It should be noted that Eqs. (l9ll-(fT2li 
are derived in the same manner as [|8] and the construction of 
the auxiliary random variable are also the same. Eqs. (ITJt and 
(O are additionally proved in this paper by using the fact that 
Alice's encoder is deterministic given the dummy randomness. 
Lemma 10: There exists e„ — > such that 

log|/C„| 
log|/C„| 

log |£„||5„| 

< /(L„,5„;r"|i^„,X2")+ne„, 
log |/C„||£„||5„| 

log |/c„||/:„||5„| 

< /(L„, r"|X„, + /(X„, X'i- Z") + 2n£„, 
log|5„| 

< /(L„,^„;r"|X„,X2") 
-/(L„,5„;Z"|if„,X^') + 4ne„, 

log |£„||^„| 

> /(xr;Z"|X„,X2")-2ne„, 
log IX I 

Proof: By using Fano's inequality, we have 

log|/C„| = H{K^) 

= I{Kn;Y'') + H{Kn\Y^) 



and 

Iog|/C„| </(i^„,X2";Z") + ne 



By using Fano's inequality and by noting that (iir„,X^) and 
(L„, Sn) are independent, we have 

l0g|£„||5„| = H{Ln,Sn) 

= /(L„,5„;y") + if(L„,5„|r") 
< I{Ln,Sn;K^,X'^\Y'')+nen 
= I{Ln,Sn;Y"\Kr.,X^)+nen. 
By using Fano's inequality, we also have 

l0g|/C„||£„||5„| = H{Kn,Ln,Sn) 

< I{Kn,Ln,Sn,X^;Y'') + nen 

and 

iog|/c„||/:„||5„| 

= H{L,,,S^\K,,) + H{K^) 

< I(Ln, + I(Kn, X^; Z") + 2nen 

where the last equality follows from the fact that X2 is a 
determined from By using the security condition and 
Fano's inequality, we have 

/(5„;Z"|if„) 

= /(5„,if„;Z")-/(i^„;Z") 

= /(5„;Z")+J(jr„;Z"|5„)-/(i^n;^") 

< /(5„;Z") + i/(A'„|Z") 

< 2ne„. (18) 
By using Fano's inequality and by using Eq. ( fTSl ). we have 

log|5„| 

< /(5„;r"|X„) + ne„ 

< /(L„,5„;F"|if„)-^(^n;^"l^n) 
-i7(L„|S'„,ii:„) +4ne„ 

-/(L„,5„;Z"|i^„,X^') + 4n£„. 

By noting that /„ is a deterministic function and by using 
Eq. ( fTSb . we have 

log|£„||A.| 

> i?(Xi"|if„,5„) 

> I{X^;Z"\Kn,Sn) 

= I{Xl\Sn;Z^\K^)^I{S^;Z^\K^) 

= I{X^;Z^\Kn,X^)-2nen. 
Finally, by noting that /„ is a deterministic function, we have 

loglXI > HiXl'\Kn,Ln,Sn) 

> I{X^;Z^\Kn,L^,Sn) 

— I{X^-^\ Z^\Kn, Ln, Sn, X2 ). 
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Lemma 11: For fixed n, let T be the random variable that 
is uniformly distributed on and is independent 

of the other random variables. Define the following random 
variables: 

= (i^„,X2",y/-\zr+i), 

Vt ~ {Ln,Sn,Ut), 

U = {Ut,T), 

V = iVT,T), 

Xi ~ XiT, 

X2 ~ X2T , 

Y = Yt, 
Z ~ Zx ■ 

Then, we have 

I{Kr,,X^-Y^) 

< nI{U,X2;Y), (19) 
/(if„,X2";Z") 

< nI{U,X2-.Z), (20) 

I{Ln, Sn] Y"\Kn, X2) 

< nI(U,V;Y\X2), (21) 

I{I^n, Ln, Sn, X2] Y^) 

< nI{U,V,X2;Y), (22) 
/(i„, y"|i^„, X^) + I{K,,, X^; Z") 

< n[I{V;Y\U,X2)+IiU,X2;Z)], (23) 

I{Lm Sn] Y"\K,i, X2) — I{Ln, Sn] Z"\Kn, X2) 

< n[I{V;Y\U,X2)-I{V;Z\U,X2)], (24) 

> n/(Xi;Z|C/,X2), (25) 

I{Xi] Z"\Kn, Ln, Sn, X2) 

> nl{Xi;Z\U,V,X2). (26) 

Proof: 

a) Proof of Eq. |79|); 

I{Kn,Xl^;Y'^) 

n 

= J2HKn,X-;Yt\Yl-') 

n 

< Y.I{Kn,X-,Yl-\Z-^,;Y^) 

n 

= Y.^{UuY,) 

t=i 

= nI{UT;YT\T) 
= nI{UT,T;YT) 
= nI{U;Y). 



6 

b) Proof of Eq. if20l ).- 

l{Kn,Xl^;Z^') 

n 

= J2^(Kn,X^;Zt\Zl\,) 

n 

< Y.I{Kn,Xl^,Yl-\Z^^^;Zt) 

n 

= Y.I{Ut;Zt) 

t=i 

= nl{UT;ZT\T) 
= nl{UT,T;ZT) 
= nI{U;Z). 

c) Proof of Eq. UZB: 

I{Ln,Sn;Y"\Kn,X^) 

n 

= J2[H{Yt\Kn,X-,Yl-^) 

-HiYt\Kn,Ln,Sn,X^,Y^~^)] 

n 

< Y.[H{Yt\X2t) 

~H{Yt\Kn, Ln, Sn,X^, Y^^', Z^_^,)] 
n 

— I{l^n, ^n, Yi -^^Vi; yt\^2t) 

n 

= Y,IiUt,Vt;Yt\X2t) 

= nIiUT,VT;YT\X2T,T) 

= nI{U,V;Y\X2). 

d) Proof of Eq. Ii22\l : 

I(,Kn, Ln, Sn, X2', Y^^) 
n 

= J2HKn,Ln,Sn,X^;Yt\Y^-^) 
t=l 

n 

^ ''^^I{Kn, Ln, Sn, X2 ,Y-I^ , Z^^i;Yt) 

t=l 

n 

t=i 

= nI{UT,VT,X2T;YT\T) 

= nI{U,V,X2;Y). 

e) Proof of Eq. ^: 

I{Ln, Sn, Y^\Kn, X^') + I{Kn, X^; Z") 

n 

= Y.^L{Ln,Sn;Yt\Kn,X^\Y,'-^) 
+I{Kn,X^i;Zt\Zl\,)] 

n 

< "y^yiiLn, Sn, Zf^i;Yt\Kn,X2, Y^ ^) 
t=l 

-1{YI ^■,Zt\Kn,X2,Z^^^) 
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+/(X„,X2",F/-^Zt|Zr+i)] 

n 
t=l 

+I{Z^+,;Yt\K„,X^,Y^-') 
—I{Y* ^■,Zt\Kn,X2,Z"_^^) 

+/(x„,x2",Yi*-^z,|zr+i)] 

n 

= ''^[I{Ln,Sn;Yt\Kn,X2,Y^ \Zt",_j) 
t=l 

+I{Kn,X^,Yt';Zt\Z^^,)] 

n 

< Sn;Yt\K^, X^, Yt\z:+,) 

t=i 

+IiKn,X^,Y*-\Z^+,:Zt)] 

= n[I{VT;YT\UT,X2T,T) + I{UT,X2T;ZT\T)] 

^ n[I{V;Y\U,X2)+I{U,X2;Z)], 

where we used Csiszar's sum identity ET\ in (a). 

f) Proof of Eq. 



I{Lm Sn', Y'^\Kn, X2) — I{Ln, Sn', Z"'\Kn, X2) 
n 

= J2^I{K,Sn;Yt\K^,X^,Y^-^) 
t=l 

~I{Ln, Sn] Zt\Kn, X2 , 1)] 
n 

Y,[IiLn,Sn;Yt\Kn,X^,Yf-') 



t=l 



I Sn 7 X2 , Y-^ ) 
— I{Y^ Zt\Kn, Ln, Sn, X2 , Zf_^l) 

^I{Ln-, Sn] Zt\Kn, X2 , Z^^^) 
n 

y~^[/(L„, Sn, Zl\_i]Yt\Kn, X2, Y]* ^) 
t=l 

~I{Ln, Sn,Yi ^]Zt\ Kn , X2 , ^t+l )] 

n 

^'^[I{Ln, Sn]Yt\Kn, X2 ,Yf 



* = 1 



~I{ZlX,;Yt\Kn,X^,Yl-') 

rt-l. 



^2\Yl-') 
-I{Yl-']Zt\Kn,Xl^,Zl\^) 

n 

J2[I{Ln, Sn] Yt\Kn, X^", ^,"+1) 
t=l 

~I{Ln, Sn] Zt\Kn, X2 ,Y^ 

- n[I{VT]YT\UT,X2T,T) - I{Vt]Zt\Ut,X2t,T)] 
= n[I{V]Y\U,X2)-I{V]Z\U,X2)], 

where (a) and (b) follow from Csiszar's sum identity ET\ . 
g) Proof of Eq. 



n 

-H{Zt\Kn,X^,X^2\Zl\,)] 



(a) 

> J2^H{Zt\Kn,X-,Yr\z:^,) 
t=i 

-H{Zt\Kn,Xu,X'2\Yt\Z^_^,)] 

n 

^ Y^I{Xu]Zt\Kn,Xl^,Yl-\Zl\,) 

n 

= ^/(Xu;Zt|C/t,X2t) 
t=l 

= nI{XiT]ZT\UT,X2T,T) 

= nI{Xi;Z\U,X2), 

where (a) follows from the fact that {Kn, X^^^, X^^^^^, 

^21'' ^2{t+iy ^1*"'' Zt+i), iXu,X2t), and Zt form Markov 
chain. 

h) Proof of Eq. HM : 

I{Xl']Z"\K^, 

n 

= ^y^SH{Zt\Kn, Ln, Sn,X2, ^t+l) 
t=l 

m -f'rn Sn, X^ , X2 , Z^^i)] 

(a) ^ 

> 2_^[^iZt\Kn, Ln, Sn, X2 ,Y^ i^t+l) 
f = l 

-H{Zt\K Sn,Xit,X2,Y^ ,Z^_^i)] 

n 

= ''^I{Xit] Zt\Kn,Ln,Sn,X2,Y^ ^ , Z^^^) 



t=l 



Y,I{Xu]Zt\Ut,Vt,X2t) 



= nI{Xi]Z\U,V,X2), 

where (a) follows from the fact that {Kn, Ln, Sn, -'^iT^' 
Xi(t+iY -^21 ' -'^2(4+1)' Yi , Zt+i), {Xit,X2t), and Zt 
form Markov chain. ■ 
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Appendix 

A. Channel Resolvability 

Since we use a result of the channel resolvability problem 
II22I in the proof of our main result, we review the channel 
resolvability problem in this appendix. For simplicity of no- 
tation, we consider the so-called one-shot case, i.e., the block 
length is n = 1. In the channel resolvability problem, for 
the input distribution Px of the channel Pz\x, we want to 
simulate the response Pz of the channel, where 



Pz{z)^Y.P^^'^)Pz\x{z\x). 
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The simulation is conducted by a deterministic map : B 
X, and uniform random number B on B. Let 



be the output distribution with map Lp. The purpose of the re- 
sol vabiUty problem is to construct a map such that D{P2;\\Pz) 
is small. 

In ifTSl . the following random coding construction of a map 
was proposed. We split the alphabet asZ? = A^ixA^2- Let 
Pvx be a distribution such that the marginal is Px- We first 
randomly generate \M.2\ codewords ui, . . . , v\j^^\ according 
to the distribution Py- We denote the generated code by 
C2. Then, for each 1 < i < \A42\, we randomly generate 
\ codewords xn, . . . , Xi\Mi\ according to the distribution 
Px\v{'\'^i)- We denote the generated code by Ci. For this 
construction we have the following lemma. 

Le}mna 12: (||T6]|) For < 0, 6*' < 1, we have 



Ec,c. [D{Pz\\Pz) 
1 



< 



4'(S\Pzlx,Pxiv,Pv) 



e'\M2\ 

where 

i^{0\Pz\x,Px\v,Pv) 

= \ogY.Pv{v)Y, 



Y,Px\v{x\v)Pz\x{z\xY^' Pz\v{z\vy 



and 



i^{e'\Pz\v,Pv) 



l°gE {^Px{x)Pz\x{z\x)'+''^ Pzi^ 



B. Proof of Lemma |9] 

a) Proof of Eq. dTTl ).- We first not the following obser- 
vation. By taking the average over randomly generated codes, 
we have 



EC0C1C2C3 [PerrU-,9)] 



V0C1C2C3 



E 

k,i,j,s,a 



|/C||Z||J||5||^| 



Py\XiX2 {T^kijsa\xikijsa,X2k) 



E 



1 



|/C||J||J||5||^| 



k,i.j,s.a 

[PY\X^X2{^^'k^Jsa\Xlk^Jsa,X2k)] 



E, 



■C0C1C2 



E 

k.i^j.s 



\1C\\I\\J\\S\ 



Py\VX2 [T^kijsa \vkijs,X2k) 



Let Tuvx2 = {y ■ {u,v,X2,y) £ T}. Then, we have 



(27) 



E, 



C0C1C2 



k,i.j,s 

{Py\VX2 (TukiVkijsX2k \'"kijs,X2k) 

+ E ^y\VX2i^u-^-.v-^-.-.-x^i^\vki3s,X2k)} 



y I 



< E, 



CqCiCs 



Sjm\\j\\s\ 



{Py\VX2 iTuk,Vkijs^2k \'"kijs,X2k) 

+ E PY\VX2{TukiV^ij,X2A'"kijs,X2k) 

+ E E ^y\VX2{7'u^.v^i-.-X2k\Vkijs,X2k) 



k,i,j,s 



+ ^ PuVX2{u,V,X2)Py\UX2{TuvX2\u,X2) 

Puvx2 (u,V,X2)Py\X2 

U,V^X2 

+ |/C||Z||J||5| PuVX2{u,V,X2)Py{TuvX2) 

< Puvx2Y{r') + \J\\S\e-"' 

+ \I\\J\\S\e-"- + \IC\\I\\J\\S\e-"', 

where we used 

PY\UX2{y\u,X2) < PY\UVX2{y\u,V,X2)e^"\ 
PY\X2{y\x2) < PY\UVX2{y\u,V,X2)e~"^, 
PY{y) < PY\uvx2iy\u,v,X2)e-"' 

for y S Tuvx2 in the last inequality. 
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b) Proof of Eq. (O; Let Ta.ux2 = {z ■ {u,X2,z) G To}- 
In a similar manner as Eq. (|27i . we have 

^CoClC2C3 



we have 



^CoC,C,Cs [D{f)] 



< E, 



CoCi 



|J| i-Pg|'^^2(7o'!Mfc,X2j"fc»'^2fc) 



fc,z 



9'|J| 



'/'(6'|Pz|!7V-X, (■h"fci,a;2fc),fv-|17X2(-|"fci.^2fc)) 



< 



e\A\' 



k^k i 

k,i ' ' ' ' 

+ |/C||X| ^ P?7X2(w,X2)Pz(7o,ua;2)} 

< Pc/X2z(V) + |/C||I|e-"°, 
where we used 

i'z(^) < Pz|c/X2(2|«,a:2)e-"° 

for z G %.ux2 in the last inequality. 

c) Proof of Eq. ( ITtI ).- By using the monotonicity of the 
divergence, we have 

Dif) = Z?(F5^||P5 X P^) 

< DiPj,jsz\\PsxPKiz) 

For each (fc, i), we use the relation 

+D{Pz\Kii-\k^^)\\Pz\ux2i-\ukt,x2k)) 
= E 7^-0(^z|A'/s(-|^'*'«)ll^2|t^X2(>fe»'2;2fe)). 

By using Lemma [T2l for input distributions Pv\ux2 {'Wki,X2k) 
and Pxi\v ™d channel Pz\XiX2^ we have 

< 



ore 



ll>{6'\Pz\UVX2^Pv\UX2^PuX2) 



1 



ol!>(S\Pz\XiX2(-\ - ^^2k) ,Pxi\V .Pv\U X2(-Wki ,X2k)) 



o'4'{S'\Pz\UVX2 {■\-,'^ki,X2k),Pv\UX2 ( ' I "fci ,a;2fc ) ) 



By taking the average over Co and Ci, and by noting 



E. 



'CoCi 



PUX2(U,X2), 



C. Proof of Lemma \8\ 

By using the Fourier-Motzkin elimination, we can show that 
{Rd,Ro,Ri,Rs) e if and only if 

Ro < HU,X2;Z), 
Ro + Rs < I{V;Y\U,X2)-IiV;Z\U,X2) 

+I{U,X2;Y), (28) 
Ri+Rs < IiU,V;Y\X2), 
Ro + Ri+Rs < I{V;Y\U,X2) 

+ mm[IiU,X2;Y),I{U,X2;Z)], 
Rs < I{V;Y\U,X2)- I{V;Z\U, X2),{29) 
Rd + Ri > I{X,;Z\U,X2), 
Rd > I{Xi;Z\U,V,X2) 

are satisfied. By adding the inequality 

i?o </([/, ^2; r), 

this inequality and Eq. ( [29] l imply that Eq. ( |28] l is redundant. 
Thus, we have 7^* C ■ 
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